1. Field of the invention
The present invention relates to an interpolation method and apparatus used, for example, for generating a frame signal from a field signal in televisions, video recorders, printers, photocopiers, and similar devices that use gray or color scale images in the image and data processing fields.
2. Prior art
Pixel density conversion technologies have become increasingly important with the devilopment of digital imaging devices. In IDTV (improved definition TV) and EDTV (enhanced definition TV), a single frame is generated by interlacing two fields in the broadcast signal and video signal, and the method of non-interlaced reproduction of these frame becomes very important.
This non-interlaced reproduction of frames can be easily accomplished using the information from one previous field when there is a correlation between the frames as in still images. When there is no precise frame correlation as in a moving image, the information from the previous field is the information for a point in time 1/60th second earlier and cannot be used for direct field interlacing. It is therefore necessary to interpolate the data for one field between the scan lines to reproduce one complete frame.
The printer engine in video printers and other video signal hard copy printers likewise records images with the same number of pixels as in a complete frame. If the input video signal is a still image, the printer can print the image directly to paper, but if the signal is a moving image, the printer engine must interpolate the information for one field to obtain the same number of pixels as in the full frame before printing the image.
Linear interpolation using the average values of the pixels in the preceding and following scan lines has conventionally been used for field interpolation. Because this interpolation method generates additional pixel data from only a few pixels, the object has been to smooth the image by increasing the number of pixels rather than to improve the resolution. The interpolated image is therefore relatively defocused or blurred compared with the original source image.
Another interpolation method has since been developed to resolve these problems with linear interpolation by using statistical properties of the image, e.g. the continuity between fields in a moving image, to obtain a higher vertical resolution and to obtain diagonal lines that are smoother than in the linearly interpolated image by using correlation detection.
This interpolation method using correlation detection is explained in further below with reference to FIG. 12.
In FIG. 12 lines A and C are scan lines from the same field input continuously to the rasterizer. Line B is the scan line that is not input in this field and which must be interpolated. If the pixel to be interpolated is pixel Bn in line B where n is the pixel number, the differences (.DELTA.1, .DELTA.2, .DELTA.3) in the three brightness levels passing through pixel Bn between lines A and C are expressed by the following equations. EQU .DELTA.1= An-1-Cn+1 EQU .DELTA.2= An-Cn EQU .DELTA.3= An+1-Cn-1
The value to be used for the interpolated pixel Bn is selected by determining which of these differences is smallest, and then applying a corresponding equation. Thus, if min.=.DELTA.1,Bn= An-1+Cn+1 /2 if min.=.DELTA.2,Bn= An+Cn /2 if min.=.DELTA.3,Bn= An+1+Cn-1 /2.
Thus, this interpolation method compares the level difference of the pixel An above and the pixel Cn below the interpolated pixel Bn with the level difference of the pixel An+1 right above and the pixel Cn-1 left below, and the level difference of the pixels An-1 left above and Cn+1 right below the interpolated pixel Bn. It is assumed that the continuity, i.e., correlation, between the images is highest in the direction in which the pixel level is lowest, and uses the average of the pixel values in this direction as the value of the interpolated pixel. (See Shashin Kogyo (Photography industry), Oct. 1989, pp. 107-108.) There is a related method that expands this concept to gray scale interpolation and expands the direction of interpolation to the right and left of these three directions (Japanese Patent Laid-Open No. H2-177683).
3. Problem to be solved
with this conventional method, however, the correlation determining the interpolation direction is evaluated by comparing the absolute values of the pixel level differences in plural interpolation directions, specifically vertically, and right and left diagonally in the above method. The highest correlation between images is determined to be in the direction of the lowest level difference, and the pixel is interpolated in this direction. This results in the following problems.
If the pixel level difference is high in all interpolation directions it should be determined that there is no real correlation and linear interpolation should be applied. But if there is even a slight difference in the pixel levels, a correlation will be wrongly detected, the average of the pixels in this wrong interpolation direction will therefore be used as the interpolated value, and pixel noise and image deterioration will result.
Furthermore, if the pixel level difference is low in all directions and there is a correlation in all directions, it should be determined that there is a real correlation between the lines and linear interpolation should be applied. But if there is even a slight difference in the pixel levels, a correlation will again be wrongly detected, and pixel noise and image deterioration will result.
FIG. 13 is an example of an image in which interpolation noise will occur. The circles drawn with a solid line are input pixels, and the dotted line circles are interpolated pixels. Two vertical black lines are input one pixel apart, and pixel Bn is obtained by interpolation. The minimum pixel level difference is obtained in the three interpolation directions shown in the figure. The pixel level difference will be low in all three directions in this example, but if there is some slight variance for any reason and the level difference is lowest in either diagonal direction, that will be selected as the interpolation direction. Bn will therefore be interpolated as a black pixel, resulting in noise.
Furthermore, if the pixel level difference is equally small in both diagonal directions compared with the vertical pixel level difference, i.e., a contradiction exists in determining the correlation from the pixel level difference, it should be determined that there is no correlation and linear interpolation in the vertical direction should be applied. With the conventional method, however, one of these diagonal directions will be selected, again resulting in image deterioration.
In general, the correlation interpolation method smoothes diagonal lines in the image and improves vertical resolution if the correlation can be correctly detected using continuous elements in the image and the interpolation direction is correct, but noise and loss of image quality result if the correlation is not correctly detected. How the correlation is evaluated therefore becomes extremely important.
To obtain an image of quality equal to a full frame signal image from a field signal requires that even nearly horizontal diagonal lines be improved. This requires at least seven directions of interpolation. When the direction closest to the horizontal is used for interpolation, nearly horizontal diagonal lines can be improved, but when the interpolation is wrong, significant noise and loss of horizontal resolution result because the interpolated pixels are horizontally separated by six pixels. Thus, when the number of pixels used to determine the correlation is large, higher precision correlation detection is required the closer the interpolation direction is to the horizontal.
Furthermore, because the angles of the interpolation lines to the vertical in the conventional method are 45, 63.4, 71.6, and 76.0 degrees, the minimum interpolation angle is 45 degrees and there is high interpolation error near the vertical direction.